Inversion in Time

Inversion of multilayer synchronous networks is a method which tries to answer questions like \What kind of input will give a desired output?" or \Is it possible to get a desired output (under special input/output constraints)?". We will describe two methods of inverting a connectionist network. Firstly, we extend inversion via backpropagation (Linden/Kindermann [4], Williams [11]) to recurrent (Elman [1], Jordan [3], Mozer [5], Williams/Zipser [10]), time-delayed (Waibel at al. [9]) and discrete versions of continuous networks (Pineda [7], Pearlmutter [6]). The result of inversion is an input vector. The corresponding output vector is equal to the target vector except a small remainder. The knowledge of those attractors may help to understand the function and the generalization qualities of connectionist systems of this kind. Secondly, we introduce a new inversion method for proving the non-existence of an input combination under special constraints, e.g. in a subspace of the input space. This method works by iterative exclusion of invalid activation values. It might be a helpful way to judge the properties of a trained network. We conclude with simulation results of three di erent tasks: XOR, morse signal decoding and handwritten digit recognition.